Segmentación y posicionamiento 3D de robots móviles en espacios inteligentes mediante redes de cámaras fijas

La presente tesis doctoral surge con el objetivo de realizar contribuciones para la segmentación, identificación y posicionamiento 3D de múltiples robots móviles. Para ello se utiliza un conjunto de cámaras calibradas y sincronizadas entre sí, que se encuentran ubicadas en posiciones fijas del espacio en que se mueven los robots (espacio inteligente). No se contará con ningún conocimiento a priori de la estructura de los robots móviles ni marcas artificiales a bordo de los mismos. Tanto para la segmentación de movimiento como para la estimación de la posición 3D de los robots móviles se propone una solución basada en la minimización de una función objetivo, que incorpora información de todas las cámaras disponibles en el espacio inteligente. Esta función objetivo depende de tres grupos de variables: los contornos que definen la segmentación sobre el plano imagen, los parámetros de movimiento 3D (componentes de la velocidad lineal y angular en el sistema de referencia global) y profundidad de cada punto de la escena al plano imagen. Debido a que la función objetivo depende de tres grupos de variables, para su minimización se emplea un algoritmo greedy, iterativo, entre etapas. En cada una de estas etapas dos de los grupos de variables se suponen conocidos, y se resuelve la ecuación para obtener el restante. De forma previa a la minimización se realiza la inicialización tanto de las curvas que definen los contornos de la segmentación como de la profundidad de cada punto perteneciente a los robots. Además se requiere la estimación del número de robots presentes en la escena. Partiendo de que las cámaras se encuentran en posiciones fijas del espacio inteligente, la inicialización de las curvas se lleva a cabo comparando cada imagen de entrada con un modelo de fondo obtenido previamente. Tanto para el modelado de fondo, como para la comparación de las imágenes de entrada con el mismo se emplea el Análisis de Componentes Principales Generalizado (GPCA). Respecto a la profundidad se emplea Visual Hull 3D (VH3D) para relacionar la información de todas las cámaras disponibles, obteniendo un contorno aproximado de los robots móviles en 3D. Esta reconstrucción de los robots proporciona una buena aproximación de la profundidad inicial de todos los puntos pertenecientes a los robots. Por otro lado, el uso de una versión extendida de la técnica de clasificación k-medias permite obtener una estimación del número de robots presentes en la escena. Tras la segmentación de movimiento y la estimación de la posición 3D de todos los objetos móviles presentes en la escena, se procede a la identificación de los robots móviles. Esta identificación es posible debido a que los robots móviles son agentes controlados por el espacio inteligente, de forma que se cuenta con información acerca de las medidas de los sensores odométricos a bordo de los mismos. Para el seguimiento se propone el uso de un filtro de partículas extendido con proceso de clasificación (XPFCP). La elección de este estimador se debe a que, dado su carácter multimodal, permite el seguimiento de un número variable de elementos (robots móviles) empleando para ello un único estimador, sin necesidad de incrementar el vector de estado. Los resultados obtenidos a la salido del XPFCP son una buena estimación de la posición de los robots móviles en un instante posterior, por lo que esta información se realimenta a la etapa de inicialización de variables, permitiendo reducir el tiempo de procesamiento consumido por la misma. Las diferentes soluciones propuestas a lo largo de la tesis han sido validadas de forma experimental utilizando para ello diferentes secuencias de imágenes (con presencia de diferentes robots, personas, diversos objetos, cambios de iluminación, etc.) adquiridas en el espacio inteligente del Departamento de Electrónica de la Universidad de Alcalá (ISPACE-UAH)

Real-time human action recognition using raw depth video-based recurrent neural networks

This work proposes and compare two different approaches for real-time human action recognition (HAR) from raw depth video sequences. Both proposals are based on the convolutional long short-term memory unit, namely ConvLSTM, with differences in the architecture and the long-term learning. The former uses a video-length adaptive input data generator (stateless) whereas the latter explores the stateful ability of general recurrent neural networks but is applied in the particular case of HAR. This stateful property allows the model to accumulate discriminative patterns from previous frames without compromising computer memory. Furthermore, since the proposal uses only depth information, HAR is carried out preserving the privacy of people in the scene, since their identities can not be recognized. Both neural networks have been trained and tested using the large-scale NTU RGB+D dataset. Experimental results show that the proposed models achieve competitive recognition accuracies with lower computational cost compared with state-of-the-art methods and prove that, in the particular case of videos, the rarely-used stateful mode of recurrent neural networks significantly improves the accuracy obtained with the standard mode. The recognition accuracies obtained are 75.26% (CS) and 75.45% (CV) for the stateless model, with an average time consumption per video of 0.21 s, and 80.43% (CS) and 79.91%(CV) with 0.89 s for the stateful one.Agencia Estatal de InvestigaciónUniversidad de Alcal

Fast heuristic method to detect people in frontal depth images

This paper presents a new method for detecting people using only depth images captured by a camera in a frontal position. The approach is based on first detecting all the objects present in the scene and determining their average depth (distance to the camera). Next, for each object, a 3D Region of Interest (ROI) is processed around it in order to determine if the characteristics of the object correspond to the biometric characteristics of a human head. The results obtained using three public datasets captured by three depth sensors with different spatial resolutions and different operation principle (structured light, active stereo vision and Time of Flight) are presented. These results demonstrate that our method can run in realtime using a low-cost CPU platform with a high accuracy, being the processing times smaller than 1 ms per frame for a 512 × 424 image resolution with a precision of 99.26% and smaller than 4 ms per frame for a 1280 × 720 image resolution with a precision of 99.77%

People re-identification using depth and intensity information from an overhead sensor

This work presents a new people re-identification method, using depth and intensity images, both of them captured with a single static camera, located in an overhead position. The proposed solution arises from the need that exists in many areas of application to carry out identification and re-identification processes to determine, for example, the time that people remain in a certain space, while fulfilling the requirement of preserving people's privacy. This work is a novelty compared to other previous solutions, since the use of top-view images of depth and intensity allows obtaining information to perform the functions of identification and re-identification of people, maintaining their privacy and reducing occlusions. In the procedure of people identification and re-identification, only three frames of intensity and depth are used, so that the first one is obtained when the person enters the scene (frontal view), the second when it is in the central area of the scene (overhead view) and the third one when it leaves the scene (back view). In the implemented method only information from the head and shoulders of people with these three different perspectives is used. From these views three feature vectors are obtained in a simple way, two of them related to depth information and the other one related to intensity data. This increases the robustness of the method against lighting changes. The proposal has been evaluated in two different datasets and compared to other state-of-the-art proposal. The obtained results show a 96,7% success rate in re-identification, with sensors that use different operating principles, all of them obtaining depth and intensity information. Furthermore, the implemented method can work in real time on a PC, without using a GPU.Ministerio de Economía y CompetitividadAgencia Estatal de InvestigaciónUniversidad de Alcal

Self-Triggered Formation Control of Nonholonomic Robots

In this paper, we report the design of an aperiodic remote formation controller applied to nonholonomic robots tracking nonlinear, trajectories using an external positioning sensor network. Our main objective is to reduce wireless communication with external sensors and robots while guaranteeing formation stability. Unlike most previous work in the field of aperiodic control, we design a self-triggered controller that only updates the control signal according to the variation of a Lyapunov function, without taking the measurement error into account. The controller is responsible for scheduling measurement requests to the sensor network and for computing and sending control signals to the robots. We design two triggering mechanisms: centralized, taking into account the formation state and decentralized, considering the individual state of each unit. We present a statistical analysis of simulation results, showing that our control solution significantly reduces the need for communication in comparison with periodic implementations, while preserving the desired tracking performance. To validate the proposal, we also perform experimental tests with robots remotely controlled by a mini PC through an IEEE 802.11g wireless network, in which robots pose is detected by a set of camera sensors connected to the same wireless network

A new framework for deep learning video based Human Action Recognition on the edge

Nowadays, video surveillance systems are commonly found in most public and private spaces. These systems typically consist of a network of cameras that feed into a central node. However, the processing aspect is evolving towards distributed approaches, leveraging edge-computing. These distributed systems are capable of effectively addressing the detection of people or events at each individual node. Most of these systems, rely on the use of deep-learning and segmentation algorithms which enable them to achieve high performance, but usually with a significant computational cost, hindering real-time execution. This paper presents an approach for people detection and action recognition in the wild, optimized for running on the edge, and that is able to work in real-time, in an embedded platform. Human Action Recognition (HAR) is performed by using a Recurrent Neural Network (RNN), specifically a Long Short-Term Memory (LSTM). The input to the LSTM is an ad-hoc, lightweight feature vector obtained from the bounding box of each detected person in the video surveillance image. The resulting system is highly portable and easily scalable, providing a powerful tool for real-world video surveillance applications (in the wild and real-time action recognition). The proposal has been exhaustively evaluated and compared against other state-of-the-art (SOTA) proposals in five datasets, including four widely used (KTH, WEIZMAN, WVU, IXMAX) and a novel one (GBA) recorded in the wild, that includes several people performing different actions simultaneously. The obtained results validate the proposal, since it achieves SOTA accuracy within a much more complicated video surveillance real scenario, and using a lightweight embedded hardware.European CommissionAgencia Estatal de InvestigaciónUniversidad de Alcal

Remote control of a robotic unit: a case study for control engineering formation

Hands-on experimentation has widely demonstrated its efficacy in engineering training, especially in control formation, since experimentation using computer-aided control system design (CACSD) tools is essential for future engineers. In this context, this article describes a case study for Control Engineering formation, based on a new lab practice for the linear and angular velocity control for a commercial P3-DX robot platform, to teach industrial control. This lab proposal includes all the stages involved in the design of a real control system, from plant identification from an open-loop test to real experimentation of the designed control system. The lab practices proposed have a twofold objective: First, it is an interdisciplinary approach that allows students to put into practice the skills from other subjects in the curriculum, facilitating the integration of knowledge. In addition, it allows increasing the motivation of the students by working with a complex and realistic plant. The proposal has been evaluated through the grades of the students, as well as the perception of both students and instructors, and the results obtained allow to confirm the benefits of the proposal.Universidad de Alcal

Towards Deep People Detection using CNNs Trained on Synthetic Images

Para solicitar sexeni

Towards dense people detection with deep learning and depth images

This paper describes a novel DNN-based system, named PD3net, that detects multiple people from a single depth image, in real time. The proposed neural network processes a depth image and outputs a likelihood map in image coordinates, where each detection corresponds to a Gaussian-shaped local distribution, centered at each person?s head. This likelihood map encodes both the number of detected people as well as their position in the image, from which the 3D position can be computed. The proposed DNN includes spatially separated convolutions to increase performance, and runs in real-time with low budget GPUs. We use synthetic data for initially training the network, followed by fine tuning with a small amount of real data. This allows adapting the network to different scenarios without needing large and manually labeled image datasets. Due to that, the people detection system presented in this paper has numerous potential applications in different fields, such as capacity control, automatic video-surveillance, people or groups behavior analysis, healthcare or monitoring and assistance of elderly people in ambient assisted living environments. In addition, the use of depth information does not allow recognizing the identity of people in the scene, thus enabling their detection while preserving their privacy. The proposed DNN has been experimentally evaluated and compared with other state-of-the-art approaches, including both classical and DNN-based solutions, under a wide range of experimental conditions. The achieved results allows concluding that the proposed architecture and the training strategy are effective, and the network generalize to work with scenes different from those used during training. We also demonstrate that our proposal outperforms existing methods and can accurately detect people in scenes with significant occlusions.Ministerio de Economía y CompetitividadUniversidad de AlcaláAgencia Estatal de Investigació

3DFCNN: real-time action recognition using 3D deep neural networks with raw depth information

This work describes an end-to-end approach for real-time human action recognition from raw depth image-sequences. The proposal is based on a 3D fully convolutional neural network, named 3DFCNN, which automatically encodes spatio-temporal patterns from raw depth sequences. The described 3D-CNN allows actions classification from the spatial and temporal encoded information of depth sequences. The use of depth data ensures that action recognition is carried out protecting people"s privacy, since their identities can not be recognized from these data. The proposed 3DFCNN has been optimized to reach a good performance in terms of accuracy while working in real-time. Then, it has been evaluated and compared with other state-of-the-art systems in three widely used public datasets with different characteristics, demonstrating that 3DFCNN outperforms all the non-DNNbased state-of-the-art methods with a maximum accuracy of 83.6% and obtains results that are comparable to the DNN-based approaches, while maintaining a much lower computational cost of 1.09 seconds, what significantly increases its applicability in real-world environments.Agencia Estatal de InvestigaciónUniversidad de Alcal